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Abstract-  In  this  paper,  new  image  processing  tools  are 
presented  for  conversion  thermal  and  visual  images, 
mainly  for  application  in  medicine  and  biology.  A  novel 
method  for  area  and  distance  evaluation  based  on 
statistical  differencing  is  discussed.  In  order  to  increase 
the  measurements  accuracy,  the  interpolation  and 
subpixel  bitmap  processing  are  chosen. 
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I.  INTRODUCTION 

Defining  regions  of  interest  precisely  is  one  of  the 
important  problem  in  thermal  and  visual  image  processing  in 
medicine  and  biology.  Once  the  boundaries  of  the 
investigated  objects  are  calculated,  thermal  parameters,  such 
as  average  temperature,  standard  deviation,  histograms,  etc. 
can  be  easily  obtained. 

Morphological  algorithms  for  finding  boundaries  with 
sufficient  accuracy  of  area  estimation  are  one  of  the  possible 
approach  [1].  Many  professional  packages  are  using  methods 
such  as  skeletoning,  opening,  dilatation  and  erosion,  although 
in  many  practical  cases,  they  are  not  successfully  applied 
when  the  image  is  colorful  with  shadows. 

Our  approach  assumes  to  use  thermal  and  visual  images 
captured  almost  simultaneously.  Visual  image  is  provides 
enough  information  to  generate  boundary  mask  that  is 
applied  to  get  the  area  of  interest  on  thermal  scene. 

II.  ADVANCED  METHODS  OF  THERMAL  IMAGE 
PROCESSING 


A.  Statistical  differencing 


One  of  the  method  for  contrast  enhancement  and  proper 
edge  filtering  is  based  on  statistical  differencing,  where  each 
pixel  value  is  scaled  by  the  standard  deviation  calculated  in 
its  neighborhood,  as  below  [1,2]: 
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where:  F(i,j,),  S(i,j,)  denote  the  original  pixel  value  and 
standard  deviation,  respectively. 

The  higher  standard  deviation,  the  lower  pixel  value.  It 
means,  that  the  part  of  the  image  with  low  color  or  luminance 
variation  will  be  enhanced,  and  more  image  details  are 
visible. 

Standard  deviation  for  WxW  window  of  the  image  is 
calculated  as: 
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where  M(m,n)  denotes  the  mean  value  of  the  part  of  the 
image  in  the  window. 

There  is  a  question  of  the  size  of  the  window  for 
calculating  standard  deviation.  In  practice,  it  can  vary  from 
4x4  up  to  32x32,  but  we  are  aware,  that  larger  windows  are 


statistically  more  significant.  This  method  is  very  sensitive 
to  the  noise  level,  and  preprocessing  in  the  form  of  low  pass 
filtering  or  averaging  is  recommended.  Lately,  Wallis 
introduced  the  modified  operator  for  statistical  differencing 
based  on  first  and  second  order  moments  [1]. 


B.  Interpolation  and  subpixel  resampling 


Due  to  the  limited  sampling  frequency  during  the  image 
acquisition  by  digital  system,  the  thermal  picture  resolution 
is  typically  low,  and  not  sufficient  for  more  accurate 
calculation.  It  is  especially  required  for  medical 
applications,  where  often  the  temperature  change  is  very 
low.  Interpolation  and  resampling  is  one  of  the  method  to 
improve  the  accuracy  of  geometrical  parameter  estimation, 
such  as  areas,  lengths,  mean  temperature  in  regions,  etc. 
There  are  many  interpolation  functions  that  can  be  used  for 
image  processing.  Sine,  square,  triangle,  bell,  cubic, 
Gaussian  functions  are  the  most  used.  Obviously,  the 
nonlinear  interpolation  gives  better  results  as  it  involves 
more  pixels  in  the  processing. 

Tradeoff  between  processing  time  and  the  order  of 
interpolation  function  has  to  be  considered.  For  thermal 
images,  which  are  typically  of  lower  resolution,  the  cubic 
interpolation  can  be  easily  implemented.  Normalized  cubic 
B-spline  interpolation  function  takes  a  form  [1,2]: 
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for  0  <  x  <  1 
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C.  Adaptive  histogram  modification 

Histogram  modification  is  typically  used  for  contrast 
enhancement.  During  image  capturing,  the  signal  from  the 
camera  is  quantized,  but  often  not  in  the  full  range. 
Additionally,  using  linear  quantization,  the  scene  is  often 
dark,  with  plenty  of  non-visible  details  due  to  the  nonlinear 
characteristic  of  human  eye  sensitivity.  Among  the  known 
methods,  the  linear  and  nonlinear  scaling  of  pixel  luminance 
or  color  components  are  mostly  used  [1].  Histogram 
hyperbolization  is  applied  to  compensate  eye  characteristics, 
however  it  is  computationally  intensive  and  it  is  difficult  to 
get  contrast  enhancement  in  real  time.  In  order  to  choose  the 
right  transfer  function  for  histogram  modification,  we  need 
to  know  the  probability  density  model  of  the  image.  In  non- 
adaptive  approach,  uniform,  exponential,  Rayleigh, 
hyperbolic  model  are  used  [1]. 

Lately,  the  adaptive  histogram  modification  has  been 
introduced  [1].  In  such  a  method  different  transfer  functions 
are  involved  for  different  pixels  in  an  image.  The  new 
approach  is  based  on  moving  window  and  the  histogram  is 
computed  for  any  pixel  neighborhood. 

III.  APPLICATION  FOR  INFLAMMATION 
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Thermal,  visual  and  radiological  image  processing  in 
parallel  brings  many  advantages.  As  shown  in  Fig.  1,  there  is 
a  visible  correlation  between  thermal  and  radiological  images 
during  pneumonia  inflammation  in  children.  The  hot  spots 
indicate  the  initial  state  of  pneumonia  [3].  The  investigations 
seem  to  be  very  practical,  while  the  thermography  is  much 
less  invasive  and  stressed  for  human  body  in  comparison  to 
radiological  tests. 

Histograms  modification  is  one  the  useful  method  of 
radiological  images  processing  for  contrast  enhancement,  as 
shown  in  Fig. 2.  One  of  the  most  helpful  functions  is  on-line 
interactive  histogram  processing,  together  with  image 
segmentation.  (Fig. 3).  Enhanced  image  is  then  binarised  by 
user-defined  threshold  level.  One  of  the  important  problems 
is  to  define  optimal  threshold  level.  This  level  can  be  found 
by  fixing  the  margins  for  the  histogram  and  eliminating  pixel 
below  and  above  certain  levels  as  shown  in  Fig.  3. 


body  will  not  reduce  significantly  thermal  effects  on  the 
skin  caused  by  internal  temperature  increase. 


Fig.  3.  Image  segmentation  by  user  defined  threshold  level 


Fig.  1 .  Thermal  and  radiological  images  of  a  child  with  pneumonia 
inflammation,  initial  state  of  the  disease,  and  after  14  days  of  treatment 


The  algorithm  is  fully  interactive,  and  the  user  can  adjust 
the  margins  for  the  histogram,  as  well  as  the  threshold  level 
for  segmentation.  The  histogram  equalizing  together  with 
binarisation  makes  possible  to  find  a  border  of  the  inflamed 
area.  The  similar  procedure  can  be  performed  for  thermal 
images.  It  could  hopefully  replace  radiological  inspections  by 
thermal  ones  in  the  future.  Such  investigations  are  the  very 
preliminary,  and  should  be  confirmed  by  more  cases,  not 
only  for  pneumonia  inflammation.  The  first  trials  are  very 
promising.  We  believe  that  thermoregulation  of  the  human 


IV.  AREA  AND  THERMAL 
SIGNATUREMEASUREMENT 

Low  temperature  variation,  and  low  thermal  and  visual 
contrast  in  many  applications  cause  the  demand  of  high 
accuracy  in  image  processing,  especially  in  biological  and 
medical  applications.  As  an  application,  we  have  chosen  the 
analysis  of  leaves  during  plant’s  growth  in  different 
conditions  -  temperature,  humidity,  pressure,  sun 
illumination,  etc. 


Fig.  4.  Thermal  of  the  plant  used  for  analysis,  non-recognized  area  of 
interest 


Fig.  5.  Original  visual  image  of  the  plant  used  for  analysis  with  shadows 
which  make  the  processing  difficult 

The  aim  of  the  research  was  to  calculate  mean 
temperature  and  standard  deviation  in  the  region  of  interest 
very  precisely  defined  on  visual  image  captured  together 
with  thermal  one.  A  few  algorithms  have  been  tested,  and 
because  of  colorful  visual  images  used,  we  proposed  novel 
approaches  based  on  separate  processing  of  color 


components  (R-red,  G-green,  B-blue)  of  the  image.  Although 
the  leaves  look  green,  the  more  precise  analysis  shows,  that  it 
is  impossible  to  make  the  proper  segmentation  by 
differentiating  using  single  color  component.  Additionally, 
the  analysis  is  very  sensitive  to  the  shading  and  external 
illumination.  In  the  general  case,  it  may  not  be  assumed  that 
the  light  position  is  always  the  same,  so  the  problem  of  e.g. 
sun  light  illumination  has  to  be  carefully  considered. 

The  following  procedure  is  proposed: 

•  Statistical  differencing  filtering  with  window  WxW 

•  Segmentation  with  threshold  TH 

•  Median  filtering 

Differencing  filtering  can  work  with  different  window’s 
size,  which  can  vary  from  10x10  to  100x100  pixels.  Having 
smaller  window  we  can  visualize  more  details,  depending  on 
their  local  statistical  characteristics.  On  the  other  hand,  larger 
window  makes  the  edge  of  the  area  of  interest  smother,  and 
some  negligible  details  are  automatically  removed.  It  is 
clearly  presented  in  Fig.  8. 

Segmentation  is  performed  globally  using  three  R,G,B 
components  of  each  pixel.  Formula  below  describes  how  the 
segmentation  is  defined  in  the  presented  method. 


0  if 


G(i,  j)>TH  +  B(i,  j)  and 
G(i,j)>TH  +  R(i,j) 


(4) 


1  otherwise 


where  S(i,j)  is  the  value  of  the  pixel  after  segmentation,  and 
R,G,B  denote  color  components  of  the  original  image. 

The  threshold  level  TH  for  segmentation  has  to  be  chosen 
very  carefully.  Using  too  low  TH  value,  we  can  artificially 
place  many  non  expected  details,  especially  on  the  border  of 
the  considered  area,  while  the  high  TH  value  causes  the 
unexpected  thinning  of  the  object.  All  these  features  of  the 
method  are  presented  in  Fig.  7-8. 


Fig.  7.  Influence  of  threshold  level  in  the  proposed  method  on  the  area  if 
interest:  777=30,  WxW=3lx3l  (left),  777=60,  WxW=  3 1x31  (right) 


Fig.  8.  Influence  of  window  size  for  statistical  differencing  on  the  area  if 
interest:  777=40,  WxW=  3 1x31  (left),  777=40,  WxW=  11x11  (right) 

Nonlinear  median  filtering  successfully  removes  the 
border  non-continuity,  pixels  or  their  groups  randomly 
placed  next  to  the  area  of  interest.  Median  filtering  was 
included  into  the  algorithm  mainly  because  of  shading 
existing  in  the  image.  Shadows  can  significantly  change  the 
results  of  area  measurements,  and  as  a  consequence,  the 
thermal  index  for  the  region  can  vary.  In  order  to  avoid 
shadows  in  the  images,  the  median  filter  ends  the  algorithm. 

Quantitative  results  confirm  the  influence  of  window’s 
size  and  threshold  level  on  accuracy  of  region  measurement. 
In  Table  I,  relative  errors  are  presented.  It  shows  the  area  of 
the  region  obtained  by  the  proposed  algorithm  referring  to 
the  area  measured  manually  (Fig.  6). 

V.  CONCLUSION 


TABLE  I 

RELATIVE  ERROR  OF  AREA  MEASUREMENT  BY  THE  PROPOSED  ALGORITHM  (IN  %) 


31x31  (WxW) 

21x21(WxW) 

1  lxl  1  (WxW) 

777=30 

0.41 

6.5 

21.6 

777=40 

12.8 

7.3 

7.0 

777=50 

23.5 

18.4 

5.9 

777=60 

34.0 

28.9 

18.4 

Fig.  6.  Visual  image  after  statistical  differencing  filter  processing  - 
histogram  equalizing,  contrast  enhancement,  more  details  visible  (left), 
region  of  interest  obtained  manually  (right) 


In  this  paper  we  present  chosen  image  processing 
methods  suitable  for  thermal  and  visual  image  processing. 
The  main  aim  was  to  use  visual  image  to  compute  thermal 
signatures  for  precisely  defined  areas  of  interest.  The 
histogram  modification  gives  many  possibilities  for  contrast 
enhancement  and  details  visualization.  As  in  medicine  and 
biology,  the  images  are  very  non-homogenous,  the 
processing  using  local  statistical  characteristics  of  the  image 
is  recommended. 
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